Many current digital cameras generally are incapable of capturing all spatial directions and all intensities of a given scene. These digital cameras cannot capture a scene such that the image reflects the full dynamic and angular extent that the photographer intended. All digital cameras have a limited field-of-view such that all directions of the scene cannot be captured. This requires a photographer to either pan around the scene taking overlapping images to capture the entire scene or select which sub-part of the scene to capture. In addition, the digital cameras cannot simultaneously capture all intensities of a scene due to the fact that the real world has a much wider range of intensities than is discernable from a single camera exposure. These real world intensities range from starlight intensity to sunlight intensity. The dynamic range of light in a scene from shadows to bright sunlight can range nearly four orders of magnitude. However, most cameras can capture only two to three orders of magnitude in a single exposure. This means that the photographer or the camera must decide which sub-part of the intensity range to capture.
One solution to this problem was given in U.S. Ser. No. 10/623,033 by Sing Bing Kang, Matthew T. Uyttendaele, Simon Winder, and Richard Szeliski entitled “System and Process for Generating High Dynamic Range Images from Multiple Exposures of a Moving Scene” filed on Jul. 18, 2003. However, this solution has several limitations. One limitation is that the technique was not designed to deal with large amounts of scene motion. Another limitation is that the technique used a per-pixel decision of which image to choose and this led to noisy results. Finally, another limitation is that the technique cannot generate panoramas.
Many other image merging techniques for HDR images assume that the input images are perfectly registered. This allows the output HDR image to be constructed as a straightforward weighted average of the radiance values of the registered inputs. The more principled approaches set the weight proportionally to the signal-to-noise of the pixel. In practice, however, perfect sub-pixel registration is difficult to achieve. This is due to several factors, such as imperfect camera calibration, parallax (especially when the inputs come from a handheld camera), and scene motion.
Some techniques address some of these issues by including per-pixel optical flow as part of the registration process. In addition, some techniques modulate the weighting function by a penalty to down-weight radiance values that were not in agreement with an automatically determined “reference” image. However, optical flow can be brittle in the presence of occlusions and the per-pixel decision built into their penalty function can lead to non-smooth results.
Therefore, what is needed is a panoramic high dynamic range (HDR) method and system of automatically combining multiple input images to create a high dynamic range image having a panoramic view. In addition, what is needed is a panoramic HDR method and system that can automatically combine multiple input images that vary in intensity range, scene orientation, or both, to create a panoramic HDR image. What is also needed is a panoramic HDR method and system that automatically merges input images to generate a final image having an extended intensity range and spatial extent.